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(57) Abstract 

A neurofuzzy device (12) is described that provides a fuzzy logic based user-machine interface (9) for optimal fitting of programmable 
hearing prosthesis (13) using a neural network (4) that generates targets to be matched by the hearing prosthesis (13) based on individual 
audiometric (1) and other relevant data (2) to the specific impairment and on the neural network (3) accumulated learning from previous 
successful fittings (8). The incorporated learning process (3) can occur on (16) or off line (14) and implements fitting rationales that can 
satisfy the needs of a general or specific clientele (8). The parameters of the programmable prosthetic device (6) are set as a group in order to 
achieve optimal matching to the targets. The user-machine interface realized by a fuzzy logic system (9) deciphers the commands/responses 
of the user (11) while listening to various stimuli and modifies the targets (5) accordingly thus, providing a closed loop system for m-situ 
interactive fitting. 
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A NEUROFUZZY BASED DEVICE FOR 
PROGRAMMABLE HEARING AIDS 

This application is a continuation-in-part of Provisional Application 
5 Serial No. 60/062,354, filed October 15, 1997, and entitled "A Neurofuzzy 
Methodology For Intelligent Hearing Prosthetics, upon which priority is claimed. 
Field of the Invention ; 

The present invention is directed towards fitting a programmable 
hearing aid by providing electroacoustic targets based on hearing impairments, on 
10 the learned information- from previous successful fittings and on user 

comments/responses while listening to different sound environments and those 
targets are to be matched by the response of the hearing aid. 
Background of the Invention : 

Programmable hearing aids offer the possibility of making choices 
15 for the values of multiple parameters, hence allowing for a very wide range of 
electroacoustic responses capable of accommodating many different kinds of 
hearing impairments. The audiologist/hearing aid practitioner using audiometric 
measurements decides on the objectives/targets that the hearing device should 
match in order to compensate for the impairment. 
20 With the advent of programmable hearing instruments, it has become 

possible to achieve near optimal matching between the electroacoustic responses 
and the targets. The targets are derived from prescriptive procedures, based on 
theoretical or empirical considerations, originally proposed for linear 
nonprogrammable hearing devices. Lately, new prescriptive formulas intended for 
25 nonlinear programmable devices such as DSLi/o and Fig. 6 are being used. 

Nevertheless, their targets are based on artificial listening environments (i.e., 
speech weighted noise) and do not adequately characterize hearing aid performance 
in realistic environments. 
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The audiologist is then often faced with user complaints that reflect 
the performance of the device in everyday environments and he/she then must 
adjust the operating parameters of the device with suboptimal tools and methods. 
Setting those parameters one at a time (e.g., AGC for the low channel, Gain for 
5 the high channel) is a suboptimal procedure because there is a great degree of 

interdependence between the parameters. Furthermore, the user responses and 
complaints are imprecise and uncertain and they need to be deciphered by the 
audiologist. 

Finally, even tools that allow for an automatic mechanism for 
10 deciphering the user responses, such as the fuzzy logic device described in US 

Patent 5,606,620, are not optimal because they directly control the individual 

parameters of the device. Because of this direct control, many contradictory 

demands on setting the parameters are not adequately resolved by the inherent 

capabilities of the fuzzy logic. Furthermore, the said system does not provide for 
15 an efficient mechanism for incorporating learning from successful fittings other 

than the manual entry of fuzzy rules. 

Summary of the Invention : 

The invention provides for a neurofuzzy device that as a first step 

in the fitting process, generates initial targets for compensating a particular hearing 
20 loss. These targets are based both on collected individualized audiometric and 

other data and on the accumulated learning from previous successful fittings. 

Starting the fitting process with efficient initial targets could significantly shorten 

the process. 

Targets could have the form of gain curves for different input levels, 
25 signal to noise curves, etc. These curves are not directly dependent on the 

particular hearing device used, rather they represent the electroacoustic response 
of an "ideal" prosthesis for the particular impairment. 

The targets are generated by a multilayer neural network which is 
a "black box" information processing system trained to generate an optimum match 
30 between a set of audiometric measurements such as the auditory thresholds and the 
corresponding best frequency and gain curves at different input levels for each 



WO 99/19779 PCT/US98/21701 

- 3 - ~ 

subject. The neural network requires a priori knowledge and acquiring it requires 
large amounts of data in order to converge to a solution. 

The required a priori knowledge is entered into the network during 
an off line training session performed during manufacturing using data collected 
from hearing aid dispensing outlets from selected geographical areas. The 
currently available prescriptive rationales such as NAL and Fig. 6 are based on 
such data and are used in fitting traditional non programmable hearing aids as well 
as starting frequency/gain target curves for programmable devices. 

The known rationales are fairly limited in scope as it is not possible 
to develop formulas based on observations of large fitting data that will adequately 
reflect the interdependences in the data. A neural network can far more 
effectively capture the essential nonlinearities of the problem. The captured 
knowledge is in the form of nodal weights in the hidden layers of the network. 

Training the network is also an ongoing process which is enabled 
after the fine tuning process that culminated in a successful fitting. The modified 
targets, resulting from the fuzzy logic based fine tuning process described later on, 
are used for retraining the neural network. Such on-line training allows for the 
neurofuzzy fitting device to be biased toward the peculiarities of a particular 
clientele. 

The fine-tuning process in this neurofuzzy methodology closes the 
loop of the fitting process. After the initial targets are generated, the settings of all 
the parameters of the hearing prosthesis are derived and transmitted to the hearing 
aid. The user is then asked to listen to different sound stimuli (e.g.., speech at 
different levels, speech and noise at different signal to noise ratio etc.) and rate the 
performance of the hearing aid using qualities of sound perception such as 
loudness, tonality, comfort, distortion, clarity etc. The fuzzy interface modifies 
the targets taking as inputs the user response as well as certain objective 
characteristics of the sound stimuli (e.g.., overall sound pressure level and signal 
to noise Ratio) and using preentered rules. 

The preentered fuzzy rules could be provided by the manufacturer 
or locally by the audiologist. The new modified targets are used for the derivation 
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of a new set of values, for all the parameters, which in turn are downloaded to the 
hearing aid. A new battery of tests is completed and the circle is repeated until 
satisfactory results are achieved. 

The hearing aid parameters are a function of both the sound 
characteristics of the input to the hearing aid and the target curves (which, at the 
limit, are identical to the electroacoustic response of the hearing aid). This 
relationship can be encoded in a neural network by pretraining it using the targets 
and the sound characteristics as inputs and the corresponding parameters as outputs. 
At the end of the fitting process, and depending on the measure of satisfaction, the 
audiologist/hearing aid professional can use the final targets for the automatic 
retraining of the neural network that generates the initial targets. 

Numerous other advantages and features of the present invention will 
become readily apparent from the following detailed description of the invention 
and the embodiments thereof, from the claims and from the accompanying 
drawings. 

Brief Description of the Drawings : 

Fig. 1 is a block diagram of the device that implements the 
neurofuzzy methodology and the interaction points (data entry blocks) with the 
hearing impaired user and the audiologist; 

Fig. 2 is a diagram of a section of the neural network that generates 
gain target values for certain frequencies based on the hearing thresholds; and 

Fig. 3 is a "flow chart" representation of the fuzzy logic system that 
modifies the targets originally generated by the neural network. The fuzzy logic 
is part of the interactive fine tuning procedure of the hearing device. 
Detailed Description of the Preferred Embodiments : 

While this invention is susceptible of embodiment in many different 
forms, there are shown in the drawing and will be described herein in detail 
specific embodiments thereof with the understanding that the present disclosure is 
to be considered as an exemplification of the principles of the invention and is not 
intended to limit the invention to the specific embodiments illustrated. 
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The device shown in Fig. 1 is a block diagram representation of the 
neurofuzzy system of the present invention used in optimizing the performance of 
a programmable hearing aid. This methodology can be realized on a personal 
computer with computer peripherals such the keyboard, the touch screen and the 
ports used as data entry and monitoring points and the sound card and the CD- 
ROM used for generating the sound signals used in the fine tuning process. 

The blocks in Fig, 1 are separate subroutines in a software package 
provided to the audiologist by the hearing aid manufacturer. The audiometric data 
1 is entered into the system via the computer keyboard and it includes, among 
others 2, audiometric threshold data at different frequencies. The above data is v 
entered into a pretrained neural network 4 and a portion of this network is shown 
in Fig. 2. 

The network generates the necessary target values that the hearing 
aid needs to match the given input level in order to compensate for the impairment 
of that particular user. Fig. 2 gives an example of target values for three different 
frequency regions. Each of these values is a function of the user audiometric data 
(only thresholds are shown here for simplicity) and the information embedded in 
the hidden neural layers. This information is put there during the training session 
3 of the neural network. Such training procedure is well known to those familiar 
with the field. 

The training of the neural net 3 is done off line 14 before the device 
(software package) is sent for use to the audiologist. In the present configuration, 
retraining of the neural net is possible after a successful fitting. 

When the level of satisfaction 8 of the performance of the hearing 
device is high enough, then retraining can take place by updating the embedded 
information in the hidden neurons of the neural net and using the set of modified 
targets 15 derived after a successful fine tuning process. Before retraining, the 
hidden neurons will produce an initial set of target curves based on the 
audiometric data 1 and other patient data 2. 

After retraining, the future target curves will be closer to the ones 
resulting from the past successful fine tuning processes. Subsequently, for a user 
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with similar audiometric data, the system will produce initial targets closer to the 
needs of this particular impairment, thus shortening the fine-tuning process. 

Every time a new set of targets is generated 5, the parameters of the 
hearing aid must be changed in order for its electroacoustic response to match the 
5 targets. Generating the appropriate parameters 6 could take the form of pattern 

matching searches or, if computation time is a constrain, implementation with a 
pretrained neural network. 

Once the parameters are generated they are sent to the hearing aid 
13 via the programmer unit 7. The programmed hearing aid 13 is worn by the 
10 user and he/she is asked to evaluate its performance while listening, to specific 
speech/sound stimuli 12. The user then quantitatively rates it in terms of certain 
sound qualities such as loudness, tonality, comfort, clarity, etc. 

The user enters his/her responses 1 1 by entering a number on a scale 
via, for example, a touch screen monitor. An example of how these responses 
15 could be interpreted are described next. If the stimuli sounds loud to him/her, then 

a high number ( 3) is entered on the membership graph for loudness scale shown 
in Fig. 2. 

The value of 3 on the x axis of said graph means that the sound 
level perceived by the user has a membership degree of 1 in the loud category and 
20 a membership degree of 0 in the soft and normal categories, as shown in the y axis 

of said graph. The membership degree numbers are the inputs to the fuzzy logic 
system 9. 

In Fig. 2, a flow chart for purposes of enabling one of skill to 
understand and practice the invention, is shown of the fuzzy logic and its operation. 

25 The membership degrees in the objective input category termed speech-level in Fig 

3, indicate that the level of the input signal to the hearing aid is very close in 
value to what a normal hearing user calls comfortable (membership degree 0.7). 
That same signal is characterized as "loud" by the hearing aid user as indicated by 
the subjective input termed "loudness" (membership degree 0.9, in loud category). 

30 This same signal is characterized as "low pitch" (membership degree 0.8) in the 
tonality scale and "almost unclear" (membership degree 0.5) in the clarity scale. 
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Given the above evaluation, it is obvious that the hearing aid in this 
example is not performing well in restoring loudness and clarity to normal levels. 

Certain adjustments to the initial targets are necessary. The 
necessary adjustments are described in the following examples of rules that the 
fuzzy logic system must implement. 

1. If low pitch or unclear, then increase high frequencies, 
/r-iir* 2. If loudness is loud, and input is normal, then decrease low, 
mid, and high frequencies. 

3. If high pitch and clear, then decrease high frequencies. 

4. If loudness is normal and input is normal, then high, mid, 
and low frequencies are O.K. 

5. If pitch is O.K. and clarity is O.K., then high frequency 
O.K. 

6. If loudness is soft and input is normal, then increase high, 
mid, and low frequencies. 

The implementation of the above examples of rules is illustrated in 
Fig. 2. The outcome of the implementation of the rules will be an increase, 
decrease or no change in the gain for the frequency/gain curve termed normal- 
target, corresponding to the target electroacoustic response of the hearing aid for 
normal level input sounds. The target curve is divided into low, mid, and high 
frequency sections for simplicity purposes. The gain of each of these sections can 
be increased, decreased, or remain unchanged, depending on the membership 
degree value assigned to it by the above inference rules. 

Rule #1 assigns the minimum of two membership degree values (that 
of the loud category in the loudness scale, and the comfortable category in the 
speech level scale) to the increase-high-frequency category of the normal-target- 
high-frequency scale. The resulting membership degree is 0.8. The same 
procedure is followed with the rest of the rules. 

If more than one membership degree is assigned to each category, 
the one with the maximum value is selected. After implementing all the rules, and 
based on the membership degree of the three frequency scales, it is clear that the 
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gain in low and mid frequencies should be decreased and the gain for the high 
frequency should remain almost unchanged. The amount of necessary change in 
gain is derived by a rule such as the center-of-gravity rule shown in the 
membership graph for the high-frequency target. 

5 The shaded area under each category (increase, O.K., decrease) 

gives a visual representation of the membership degree. The center of gravity of 
the shaded area is closen to?3^than to -3 in the x axis (close to 1). The centroid is 
a representative crisp value which indicates how many decibels of gain must be 
added to the high frequency target. Thus, Fig. 3 indicates a crisp value of 3dB 

10 should be subtracted from the^mid and low frequency gain of the target curve.. 

When a battery of different listening tests and evaluations is 
completed using the new modified targets, the fine tuning procedure will continue 
until the audiologist decides that a satisfactory level of performance is reached. 
The satisfaction meter 8 can be based on the weighted accumulated changes of the 

15 target curves. Once a converging value is reached, this could be indication of 

having reached a point of diminishing returns in the fine tuning process. 

Although modifications and changes may be suggested by those 
skilled in the art, it is the intention of the inventor to embody within the patent 
warranted hereon all changes and modifications as reasonably and properly come 

20 within the scope of his contribution for the generation and the adjustments to the 

initial targets as are necessary. 
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From the foregoing, it will be observed that numerous variations and 
modifications may be effected without departing from the spirit and scope of the 
invention. It is to be understood that no limitation with respect to the specific 
apparatus illustrated herein is intended or should be inferred. It is, of course, 
intended to cover by the appended claims all such modifications as fall within the 
scope of the claims. 
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What is Claimed : 

1 , A device for use with a programmable hearing prosthesis that 
sets the operational characteristics of said prosthesis, said device comprising: 

neural network means for generating initial targets addressing 
5 at least one aspect of the hearing impairment and at least one listening 

environment; 

means for~entering audiological and other relevant data that 
characterize aspects of the impairment and of the user; 

fuzzy logic means incorporating rules generating commands 
10 for modifying said initial targets based on user provided responses while listening 

to different sound environments; 

neural network means for receiving said commands from said 
fuzzy logic means to modify said initial targets; 

data processing means for using said modified targets to 
15 optimize all the hearing aid parameters of said prosthesis as a group; 

means for downloading said operational parameters to the 

hearing prosthesis; 

means for generating different sound stimuli for the in situ 
evaluation of the performance of the hearing prosthesis; 
20 means for receiving the user responses to be used by the said 

fuzzy logic means; 

means for the on-line retraining of the neural network after 

successful fittings; 

means for the off-line retraining of the neural network from 
25 off site collected data on successful sittings; and 

means for entering fuzzy processing rules to said fuzzy logic 

means. 

2. A device as in claim 1 which includes a processor and a 
plurality of instructions wherein the neural means is implemented when the 

30 processor executes the instructions. 
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3, A programming device usable to program a hearing aid 

comprising: 

a neural processing network for generating a plurality of 
output indicia indicative, at least in part, of parameters for the hearing aid to be 
programmed; and 

a fuzzy logic processing system, coupled to the output indicia 
for modifying same tcFpbduce revised output indicia. 

4. A device as in claim 3 which includes post-processing 
circuitry coupled between the network and the processing system. 

* 5. ^system for programming a selected hearing aid comprising: 
input circuitry for receiving indicia of an individual's hearing 

characteristics; 

a multi-level neural processing network with an input port 
that is coupled to the input circuitry, for processing the indicia and for generating, 
at an output port, at least one parameter value usable for programming the selected 
hearing aid wherein the value has been established by the network in response to 
both the input indicia and pre-established characteristics of the selected hearing aid. 

6. A system as in claim 5 which includes a processor and a 
plurality of instructions wherein the network is implemented by the processor's 
execution of the plurality of instructions. 

7. A system as in claim 6 wherein the processor is coupled to 
the input circuitry. 

8. A system as in claim 6 which includes instructions for 
responding to the feedback of the individual in response to an initially established 
parameter and for modifying that parameter in response thereto. 

9. A system as in claim 6 which includes a display unit, coupled 
to the processor and additional instructions for presenting the expected response 
characteristics of the hearing aid as a result of the generated parameter value. 

10. A system as in claim 5 wherein the network incorporates a 
plurality of interconnected node elements wherein some of the elements are 
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associated with one layer and others of the elements are associated with a different 
layer. 

11. A system as in claim 10 wherein at least some of the nodes 
are implemented by pluralities of instructions executed by at least one processor. 
5 12. A system as in claim 10 which includes a plurality of 

processors wherein at least one node is implemented by one processor executing 
a first set of instructions and another node-is "implemented by another processor 
executing another set of instructions. 

13. A system as in claim 6 which includes a fuzzy logic system, 
10 for receiving user feedback signals and for-altering the at least one parameter in 

response thereto. 

14. A system as in claim 5 which includes post-processing 
circuitry, coupled to the output port, for modifying network outputs. 

15. A system as in claim 14 which includes feedback responsive 
15 circuitry, coupled to the post-processing circuitry, for modifying programming 

parameter values in response to use feedback. 

16. A programmer for a hearing aid comprising: 

a neural network for generating initial targets addressing at 
least one aspect of a user's hearing impairment and at least one listening 
20 environment; 

circuitry, coupled to the network for entering audiological and 
other relevant data that characterize aspects of the impairment and of the user; 

a fuzzy logic processor incorporating rule generating 
commands for modifying said initial targets based on user-provided responses while 
25 the user is listening to different sound environments; 

circuitry and prestored instructions for receiving sid 
commands from said fuzzy logic processor to modify said initial targets; and 

processing circuitry and associated instructions for using said 
modified targets to optimize at least some of the hearing aid parameters of said 
30 prosthesis as a group; 
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circuitry for downloading said operational parameters to the 

hearing prosthesis. 

17. A programmer as in claim 16 which includes: 
processing circuitry and associated instructions for using said 

5 modified targets to optimize at least some of the hearing aid parameters of said 

prosthesis as a group. 

18. A programmer as in claim 17 which includes: 

means for generating different sound stimuli for the in situ 
evaluation of the performance of the hearing prosthesis; 
10 means^for receiving the user responses to be used by the^said 

fuzzy logic processor; 

means for the on-line retraining of the neural network after 

successful fittings; 

means for the off-line retraining of the neural network form 
15 data collected off-site after successful fittings;, and 

means for entering fuzzy processing rules to said fuzzy logic 

processor. 
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